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D scription 

[0001] This invention relates to electronic music and, 
more particularly, to an electronic music system with 
which a non-musician can produce melodic, creative 
music without knowledge of music theory or the ability 
to play an instrument or keep time. 
[0002] Electronic keyboards and other electronic mu- 
sical instruments are known. Many electronic keyboard 
instruments generate digital data compatible with the 
Musical Instrument Digital Interface (MIDI) standard. 
Many electronic musical instruments also provide an au- 
tomatic accompaniment or background which is played 
by the instrument at the performer's request. With many 
-kaown electronic musical instruments, in order to make 
organized melodic sounds which would be considered 
"music", the performer must actually be able to play the 
instrument or at least be able to strike the instrument's 
"actuators" (i.e., keys of a music keyboard, strings of a 
stringed instrument such as a guitar, etc.) in lime", 
meaning in some order appropriate for the time signa- 
ture and tempo of the piece of music, song, or melody 
being played by the performer on the instrument. With 
other known musical instruments, the performer makes 
music by keying a pre-recorded melody on and off 
whenever desired. 

[0003] U.S. Patent No. 5,099,738 to Hotz discloses a 
MIDI-compatible electronic keyboard instrument that 
does not allow the musician to strike a wrong note. Dur- 
ing the interval of time in which a particular chord is be- 
ing played, the instrument generates, in response to the 
musician's depression of any key, a "correct" note (i.e., 
pitch) in that chord or a "correct" note in a scale which 
is compatible with that chord. Like other known electron- 
ic musical instruments, the time when notes are played 
are determined entirely by when the musician depress- 
es a key on the keyboard. If the musician does not or 
cannot depress the keys at appropriate times, the result 
will be "correct" notes played in an unorganized, random 
sequence. The musician thus is given "creative input" 
as to the time when notes are played but does not have 
the option of playing an incorrect chord or note. 
[0004] U.S. Patent No. 5,393,926 to Johnson disclos- 
es a virtual MIDI guitar system. The system has a per- 
sonalxomputer which utilizes a CD-ROM player to play 
back a stored audio and video accompaniment selected 
by a user. The accompaniment is a recording of a song 
with the guitar track omitted. The personal computer 
stores the guitar track of the song. The guitar has strings 
and a tremolo bar, and a user's manipulation of the 
strings and tremolo bar sends digital data to the person- 
al computer. The personal computer uses that data to 
access and play back relevant portions of the guitar-only 
track, as described below. The personal computer also 
mixes the guitar track with the audio track from the CD- 
ROM player and broadcasts it through speakers while 
at the same time displaying the video image on a mon- 
itor connected to the personal computer. The guitar-only 
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track contains all of the guitar notes in the sequence in 
which they are to be played, and it is partitioned into a 
sequence of frames. The guitar player is able to gener- 
ate only those notes that are within the current frame 

5 and only in the order in which they appear in the current 
frame, "current" being determined by a clock variable 
which tells the elapsed time since the song began. The 
pace at which the notes are played within the current 
frame is determined by when the user strikes the strings 

10 such that the user may be able to get somewhat out of 
alignment with the accompaniment in any particular 
frame and may have some flexibility to modify or exper- 
iment with the timing of the notes of the guitar track with- 
in a frame. If the player does not play the guitar during 

15 a period associated with a given frame, none of the mu- 
sic within that frame will be generated. Striking strings 
of the guitar thus causes an otherwise silent, running, 
pre-recorded guitar-only track to be heard, and the gui- 
tar thus essentially operates as an on/off or play/silent 

20 button for the pre-recorded guitar track. 

[0005] U.S. Patent No. 5,074,182 to Capps et al. dis- 
closes a guitar-like instrument with encoded musical 
material that includes a plurality of multi-part back- 
ground songs and a plurality of solo parts or "riffs" that 

25 harmonize with the background songs. A read only 
memory (ROM) in the instrument stores a program and 
the encoded musical material. Once the user has se- 
lected and started a background song, the user can trig- 
ger a guitar riff by operating some switches on the in- 

30 strument. Manipulating the switches thus causes one of 
a plurality of pre-stored riffs to play over the selected 
background song. 

[0006] US Patent No. 5391829 describes an electron- 
ic musical instrument including automatic performance 

35 and the possibility of the user interacting with the instru- 
ment such that he can play e.g. rhythm instrument 
sounds generated by the electronic instrument ad-lib 
while the corresponding instrumental sound is muted 
from the accompaniment. 

40 [0007] US Patent No. 5403970 describes a joy-stick 
interacting with an electronic musical instrument to con- 
trol all kinds of tonal, but not rhythmical, parameters. 
[0008] It is an object of the invention to provide an 
electronic music system that non-musicians can use to 

45 generate melodic, creative music in real-time without 
knowledge of music theory and without the ability to play 
an instrument or keep time. 

[0009] It is also an object to allow the user of the sys- 
tem to create and play easily a non -pre-recorded solo 
50 over a pre-recorded background or accompaniment 
track. 

[001 0] It is a further object to allow the user of the sys- 
tem to create solos without the need to strike actuators 
in time or otherwise physically establish and maintain 
55 the timing of the notes of the solo. The system does not 
require the user to, for example, keep a steady beat. 
[0011] It is still another object to provide the user of 
the system with one or more simple controllers (e.g., a 
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joystick which can have one or more buttons) for manip- 
ulating the system and generating the solo in real-time. 
[0012] The present invention is defined by the ap- 
pended claim 1 , with advantageous embodiments given 
by the dependent claims 2-34. 5 
[0013] All of the complexity associated with creating 
music is placed in the system of the invention. A user of 
the system need not know anything about music or mu- 
sical instruments to create music with the system. Ex- 
cept for the background track, the music generated by 10 
the system under the control of the user is produced in 
real-time and it is not simply a play back of a pre-record- 
ed solo track. 

[0014] Thus, the invention provides an electronic mu- 
sic system having an input mechanism, computer stor- is 
age" media, a rhythm generator, a pitch selector, and a 
sound generator. The input mechanism provides 
rhythm-related input signals and pitch-related input sig- 
nals, for example, in response to a user's manipulations 
of it. In one embodiment, the user manipulates the input 20 
mechanism to create and play music (e.g., a solo line) 
over one of a plurality of user-selectable musical back- 
ground or accompaniment tracks. In general, a solo 
means a composition or section for one performer. A so- 
lo can be a musical line of single pitches sounded one 25 
after another (i.e., a melody), or it can be a line that has 
intervals (i.e., two different pitches sounded at the 
same) and/or chords (i.e. , three or more different pitches 
sounded simultaneously) as well as, or in place of, just 
single pitches. (In general, whenever "melody" is used 30 
hereinafter, it should be taken to mean a melody or a 
solo, as those two words have been defined above. Al- 
so, "solo" includes "melody" by definition.) 
[0015] The computer storage media (e.g., computer 
memory such as RAM, a computer hard disk drive, and/ 35 
or a CD-ROM drive with a CD-ROM therein) contain the 
user-selectable accompaniment tracks and a plurality of 
rhythm blocks. Each rhythm block defines, for at least 
one note, at least a time at which the note should be 
played. A rhythm block also can specify a duration and 40 
a loudness for the note, and if these are not specified 
by the rhythm block, default or predetermined values are 
used. The computer storage (e.g., RAM) also stores at 
least the portion of the solo created over some time in- 
terval in the immediate past. It preferably stores all of 45 
the user's solo line automatically in real-time as it is cre- 
ated by the user. This "past solo" information is used by 
the pitch selector in selecting the next pitch to output. 
[0016] The rhythm generator receives the rhythm-re- 
lated input signals from the input device, selects one of so 
the rhythm blocks from storage based on the rhythm- 
related input signals, and then outputs a "play note" in- 
struction which indicates the time at which to play the 
note as defined by the selected rhythm block. The pitch 
selector receives the pitch-related input signals from the 55 
input device and selects an appropriate pitch based on 
the pitch-related input signals, harmony and metric data 
in the user-selected accompaniment track, and the "past 



solo" information. The pitch selector then outputs that 
appropriate pitch. The sound generator receives both: 
(i) the user-selected accompaniment track; and (ii) the 
user-created solo track which includes timing informa- 
tion from the rhythm generator and pitch information 
from the pitch selector. The sound generator then gen- 
erates a representative audio signal. 
[0017] In a preferred embodiment, the input device is 
a joystick having a base, a movable handle, and one or 
more buttons. The buttons can be used by the user to 
tell the electronic music system "play" and to perform 
certain musical effects such as: sustain the current note; 
play a particular riff; repeat the lick just played; alter the 
timbre; bend the pitch; play a chorci instead of a single 
note; add a dynamic accent; and/or add articulation. 
Moving the joystick's handle along the forward/back- 
ward axis can provide the rhythm-related input signals, 
and the right/left axis can be associated with the pitch- 
related input signals. For example, pulling the handle all 
the way backward can be an indication to the electronic 
music system to generate notes with the lowest rhythmic 
activity (e.g., whole notes), and pushing it all the way 
forward can mean generate the highest-activity notes 
(e.g., sixty-forth notes). When the handle is between 
these two extremes, rhythmic activity between the two 
extremes is generated. Also, moving the handle all the 
way to the right can correspond to the highest possible 
pitch, and the leftmost position can mean the lowest 
possible pitch, with a position therebetween meaning a 
pitch between the highest and lowest pitches. The user 
can manipulate the joystick handle and quickly and eas- 
ily switch rhythms and pitches. These two simple move- 
ments alone (back and forth, and left and right) allow the 
user to create a solo with rich and varied rhythmic and 
tonal qualities. 

[0018] In some embodiments, the electronic music 
system includes one or more speakers for broadcasting 
the audio signal from the sound generator. An amplifier 
generally must be used to amplify the audio signal be- 
fore it is broadcast through the speaker(s). 
[0019] Also, in a preferred embodiment of the elec- 
tronic music system according to the invention, a pro- 
grammed computer performs the functions of the 
rhythm generator and the pitch selector. The pro- 
grammed computer can also perform the functions of 
the sound generator, or the sound generator can be a 
MIDI tone generator separate from the computer. The 
speaker(s) and/or the amplifier can be internal to the 
computer as well. 

[0020] Preferably, the invention features an electronic 
rhythm generation system having an input device, a 
computer storage medium, and a rhythm generator. The 
input device generates rhythm -related input signals in 
response to manipulations of the input device by a user. 
The computer storage medium has a plurality of rhythm 
blocks wherein each rhythm block defines, for at least 
one note, at least a time at which the note should be 
played. The rhythm generator receives the rhythm-re- 
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lated input signals from the input device, selects one of 
the rhythm blocks from the computer storage medium 
based on the rhythm-related input signals, and outputs 
an instruction to play the note at the time defined by the 
selected rhythm block. 

[0021] Preferably, the invention involves an electronic 
pitch selection system which comprises an input device, 
computer storage media, and a pitch selector. The input 
device generates pitch-related input signals in response 
to manipulations of the input device by a user. The com- 
puter storage media has a plurality of user-selectable 
musical accompaniment tracks, and stores at least the 
pitches selected by the system over a predetermined 
time interval in the immediate past. The pitch selector 
receives the pitch-related input signals from the input 
device, and then selects an appropriate pitch based on 
the pitch-related input signals, the user-selected musi- 
cal accompaniment track, and the stored pitches. The 
pitch selector outputs that appropriate pitch. 
[0022] Preferably, the invention involves an electronic 
system for processing data representative of a musical 
score to modify automatically the score by adding instru- 
ment-specific performance parameters or musical orna- 
mentation. 

[0023] Some preferred embodiments of the invention 
will now be described by way of example only and with 
reference to the accompanying drawings, in which: 

Brief Description of the Drawings 

[0024] In the drawings, like reference characters gen- 
erally refer to the same parts throughout the different 
views. Also, the drawings are not necessarily to scale, 
emphasis instead generally being placed upon illustrat- 
ing the principles of the invention. 
[0025] FIG. 1 is block diagram of a computer-assisted 
real-time music composition system which uses a sim- 
ple controller in accordance with the invention. 
[0026] FIG. 2 is a simplified block diagram of a com- 
puter in which the present invention can be embodied. 
[0027] FIG. 3A is a perspective view of a computer 
joystick for use as an input device/controller of the sys- 
tem in accordance with the invention. 
[0028] FIG. 3B is also a perspective view of the joy- 
stick showing the meaning of various movements in one 
embodiment of the invention. 

[0029] FIG. 4A is a simplified flowchart of a set-up pro- 
cedure a user goes through before generating music 
with the system of the invention. 
[0030] FIG. 4B is a more complete depiction of the 
set-up procedure. 

[0031] FIG. 4C is a data path diagram showing which 
functional blocks of the system according to the inven- 
tion use what data/variables. 

[0032] FIG. 5 is a high-level flowchart of the opera- 
tions performed by the system of the invention after set- 
up is complete. 

[0033] FIGS. 6A, 6B, 6C, and 6D each shows an ex- 



ample of the rhythm block data structure. 
[0034] FIGS. 7A and 7B each shows an example of 
the rhythm style data structure. 
[0035] FIG. 8 is a high-level flowchart of the steps per- 
5 formed by the rhythm generator functional block of the 
system of the invention. 

[0036] FIG. 9 is a high-level flowchart of the steps per- 
formed by the pitch selector functional block of the sys- 
tem of the invention. 
to [0037] FIG. 10 is a detailed functional block diagram 
of the computer-implemented system according to the 
invention. 

Description 

75 

[0038] Referring to FIG. 1 , a system 10 according to 
the invention generates music in real-time in response 
to a user's manipulation of one or more simple control- 
lers/input devices 12 such as a joystick. The system 10 

20 includes a computing device 1 4, a sound generator 1 6, 
and one or more speakers 1 8. The computing device 1 4 
typically is a personal-type computer running programs 
which generate in real-time digital data representative 
of music in response to the joystick 12 manipulations. 

25 The data is then turned into audible music by the com- 
bination of the sound generator 16 and the speaker(s) 
18. 

[0039] The system 10 is an electronic music system 
that is designed for non-musicians but which can be 

30 used by anyone that wants quickly and easily to gener- 
ate melodic, creative music in real-time. The user is not 
required to have any knowledge of music theory or the 
ability to play an instrument or keep time. All the user 
needs to know how to do is to manipulate the joystick 

35 12. Other equally simplistic input devices can be used 
in place of the joystick 12 to create music including, for 
example, a mouse, a game pad, a tracball, a MIDI key- 
board, a MIDI guitar, other MIDI instruments, any of a 
variety of spatial sensors that can track hand/body mo- 

40 tion through the air, one or more switches such as the 
up/down volume touch buttons on an electronic car ra- 
dio, or any combination of such input devices. The us- 
er's manipulations of the input device (e.g., joystick 12) 
send actuator signals (e.g., changes in the positions of 

45 buttons or continuous controllers like the axes of a joy- 
stick's handle) which cause the system 10 to generate 
and play a non-pre-recorded melody over a user-select- 
ed pre-recorded accompaniment/background track. 
[0040] All of the complexity associated with creating 

50 music from a traditional or known instrument has been 
incorporated into the system 10 of the invention. The 
system 10 relieves the user of the burden of having to 
learn to play a traditional or known instrument. The sys- 
tem 10 provides the user with a simple controller/input 

55 device (e.g., the joystick), and the user thus is free to 
concentrate solely on the music itself. The user does not 
have to worry about instrument-playing technique, be- 
ing in tune, playing in time, etc. The system 10 of the 
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invention has been designed to handle all of those con- 
cerns. Even though the system 10 uses a very simple- 
to-operate interface (e.g., the joystick 12) and the user 
need not have any special musical abilities or knowl- 
edge, the user generally is not limited in the type, style, 
or variety of music that he can produce with the system 
10 of the invention. The system 10 allows a user to do 
essentially anything that can be done with any traditional 
or known instrument. 

[0041] Referring still to FIG. 1, the function of the 
sound generator 16 is to generate signals representa- 
tive of audible music, and this can be accomplished by, 
for example, synthesis or sample playback. The elec- 
tronic hardware needed to generate these signals can 
reside on a card plugged into the computer 1 4, or it can 
beun'sTseparate box external to the computer 14. Also, 
in the case of synthesis, the signal generation can be 
performed either in hardware or entirely by software run- 
ning on the computer 14. The sound generator 16 can 
be, for example, a MIDI tone generator or other synthe- 
sis device. The signals generated by the sound gener- 
ator 16 generally must be amplified and broadcast by 
the speakers 18. The amplification and broadcasting 
can be accomplished by, for example, hardware internal 
to the computer 1 4 or hardware external to the computer 
14. 

[0042] The computer 14 can be any personal-type 
computer or workstation such as a PC or PC-compatible 
machine, an Apple Macintosh, a Sun workstation, etc. 
The system 10 was developed using a Macintosh Pow- 
erbook 540c computer with 12 megabytes of RAM and 
the MAC/OS 7.5.1 operating system, and the computer 
programs for implementing the functionality described 
herein were written in the C++ programming language. 
In general, any computer could be used as long as it is 
fast enough to perform all of the functions and capabil- 
ities described herein without adversely affecting the 
quality of the generated music. The particular type of 
computer or workstation is not central to the invention. 
In fact, the music composition system according to the 
invention can be implemented in a variety of ways in- 
cluding an all-hardware embodiment in which dedicated 
electronic circuits are designed to perform all of the func- 
tionality which the programmed computer 14 can per- 
form. 

[0043] Referring to FIG. 2, the computer 14 typically 
will include a central processor 20, a main memory unit 
22 for storing programs and/or data, an input/output (I/ 
O) controller 24, a display device 26, and a data bus 28 
coupling these components to allow communication 
therebetween. The memory 22 includes random access 
memory (RAM) and read only memory (ROM) chips. 
The computer 14 typically also has one or more input 
devices 30 such as a keyboard 32 (e.g. , an alphanumer- 
ic keyboard and/or a musical keyboard), a mouse 34, 
and the joystick 12. In a preferred embodiment, the sys- 
tem 1 0 includes the single joystick 1 2, the alphanumeric 
keyboard 32, and the mouse 34. In general, the joystick 



12 is used by the user to create music with the system 
10, and the alphanumeric keyboard 32 and mouse 34 
are used by the user to setup and configure the system 
10 prior to the actual creation of music with the system 

5 10. 

[0044] The computer 1 4 typically also has a hard drive 
36 with hard disks therein and a floppy drive 38 for re- 
ceiving floppy disks such as 3.5 inch disks. Other devic- 
es 40 also can be part of the computer 1 4 including out- 

io put devices (e.g., printer or plotter) and/or optical disk 
drives for receiving and reading digital data on a CD- 
ROM. In the disclosed embodiment, one or more com- 
puter programs written in C++ define the operational ca- 
pabilities of the system 10, as mentioned previously. 

is These programs can be loaded onto the hard drive 36 
and/or into the memory 22 of the computer 14 via the 
floppy drive 38. In the disclosed embodiment, the exe- 
cutable version of the C++ programs are on the hard 
drive 36, and the music composition system 10 accord- 

20 ing to the invention is caused to run by double-clicking 
the appropriate icon. In general, the controlling software 
program(s) and all of the data utilized by the program(s) 
are stored on one or more of the computer's storage me- 
diums such as the hard drive 36, CD-ROM 40, etc. In 

25 general, the programs implement the invention on the 
computer 14, and the programs either contain or access 
the data needed to implement all of the functionality of 
the invention on the computer 14. 
[0045] Focusing back on some of the more central as- 

30 pects of the invention, the input device/controller (e.g., 
the joystick 12) which a user of the system 10 manipu- 
lates to create music preferably allows the user to indi- 
cate to the computer 14 a variety of information. Refer- 
ring to FIGS. 3A and 3B, in a preferred embodiment, this 

35 is accomplished by the joystick 12 being movable in at 
least four directions 42, 44, 46, 48 and having at least 
three buttons 50, 52, 54. 

[0046] In the disclosed embodiment, pulling the han- 
dle of the joystick of FIG. 3B in the backward direction 

*o 42 indicates to the computer 14 that the user wants to 
play fewer notes over time (e.g., half notes as opposed 
to eighth notes) in the given time signature, and pushing 
it forward 44 is an indication to play more notes over 
time (e.g., thirty-second notes as opposed to quarter 

45 notes). The handle of the joystick 12 moves from its 
backwardmost position to its forwardmost position 
through a series of rhythmic values starting with notes 
having the lowest rhythmic activity (e.g., whole notes) 
at the backwardmost position and going all the way to 

50 notes having the highest rhythmic activity (e.g., sixty- 
forth notes) at the forwardmost position. The user gen- 
erally can create any rhythmic output by moving the han- 
dle of the joystick back and forth. The selection of the 
end points of this series and the number and type of 

55 notes in between the two end points generally is made 
by the system designer/programmer. There are a large 
number of possible series or continuums, and the sys- 
tem usually selects one or more particular series auto- 
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matically without any user involvement. The system typ- 
ically will select one or more series of rhythm values 
based on the user-selected (or default) accompaniment 
and/or style of music. These rhythm continuums and the 
selection of them will become clear hereinafter from dis- 
cussions about the "rhythm generator" aspect of the 
system 10 according to the invention. 
[0047] Continuing with the current example, pushing 
the handle of the joystick to the left 46 indicates to the 
computer 1 4 that the user wants to play notes of a lower 
pitch (i.e., frequency or tone), and pushing it in the right 
direction 48 is an indication to play higher-pitched notes. 
As with the rhythmic values, the joystick 1 2 moves from 
its leftmost position to its rightmost position through a 
series-of pitches starting with a lowest-pitched note at 
the leftmost position and going all the way to a highest- 
pitch note at the rightmost position. The user can pro- 
duce virtually any combination of pitches by manipulat- 
ing the handle side to side. The program running on the 
computer 14 generally determines the notes in the se- 
ries, and the determination typically is based on the se- 
lected accompaniment and/or style of music. 
[0048] Referring to FIG. 3A, in the disclosed embod- 
iment, the joystick 12 has at least a play button 50, a 
sustain button 52, and a lick repeat button 54. 
[0049] The play button 50 is used by the user to indi- 
cate to the computer 1 4 when to start creating and play- 
ing the melody under the user's joystick control. The us- 
er must depress and hold the play button 50. Depressing 
the play button 50 enables the "rhythm generator" (dis- 
cussed hereinafter). As alluded to previously, in the dis- 
closed embodiment, the output of the rhythm generator 
is determined by the forward/backward position of joy- 
stick 12 (FIG. 3B). The user is only allowed to create 
and play a melody after the accompaniment has been 
started, and the user preferably starts the accompani- 
ment by using the mouse 34 and/or alphanumeric key- 
board 32 to click on a graphic start button on the monitor 
26 of the computer 14. 

[0050] The sustain button 52 is used by the user to 
indicate to the computer 1 4 that the note currently play- 
ing (or the next note played) should be sustained or held 
such that it continues to sound. That is, the current note 
is maintained for an extended period of time. This is sim- 
ilar to a vocalist "holding a note". The note ends when 
the user releases the sustain button 52. 
[0051] The lick repeat button 54, when depressed, 
causes the system 1 0 to repeat a particular collection of 
notes previously played. This button 54 is useful if the 
user has just created a particularly pleasing "lick" or "riff" 
(which generally is a catchy collection of several notes) 
and wants to repeat it automatically without having to 
figure out and re-enact exactly what she just did with the 
joystick 1 2 to create the lick in the first instance. The lick 
stops repeating when the user releases the lick repeat 
button 54. The point in history at which the system 10 
demarcates the beginning of the lick is randomly or al- 
gorithmically determined by the computer program. The 



length of the repeated segment is typically a few beats 
or less, as described hereinafter under the "licker" sec- 
tion. 

[0052] The discussion of this algorithm brings up the 

s important point that the programmed computer 14 is a 
digital processing device which is capable of storing in 
digital format some or all of the data it generates and 
outputs to the sound generator 16 (FIG. 1). That is, it 
can, and does, store (e.g., on the hard drive 36, in mem- 

io ory 22, etc.) the data representative of the melody the 
user is creating as it is being created. This capability is 
what allows the user to repeat a lick with the lick repeat 
button 54. The computer 14 generally stores the last ten 
notes of the melody, although this parameter is config- 

75 urable and can be set to store more or less notes. 
[0053] Having described the environment in which the 
invention operates and generally the overall purpose 
and functionality of the music composition system 10 of 
the invention, the following is a more detailed descrip- 

20 tion of the invention. 

SETUP: 

[0054] Referring to FIGS. 1, 2, and 4A, the pro- 

25 grammed Computer 14 of the system 10 takes the user 
through a configuration or setup procedure before the 
user is allowed to create music with the system 10. In a 
preferred embodiment, the input devices 30 used by the 
user to configure or setup the system 10 are the key- 

30 board 32 and/or the mouse 34. After the setup is com- 
plete, the user generally uses the joystick 12 (or other 
similarly simple-to-operate input device) to create music 
with the system 10. During the setup stage, the pro- 
grammed computer 14 allows the user to select a par- 

35 ticular background or accompaniment track (step 68) 
from a list of a plurality of possible tracks. In a preferred 
embodiment, the background tracks are stored either as 
MIDI files or as audio files. In general, MIDI files are 
small (i.e., do not take up a large amount of space on 

40 the storage medium such as the memory 22, the hard 
drive 36, or the CD-ROM 40) and audio files are com- 
paratively large. If the tracks are MIDI, the selected track 
typically will be loaded into the memory 22 of the com- 
puter 14 from its hard drive 36 or CD-ROM 40, for ex- 

45 ample. If the tracks, however, are audio, the selected 
track typically will not be loaded into memory 22 and will 
instead be streamed off of, for example, the hard drive 
36 or the CD-ROM 40 as needed during the user's per- 
formance. 

50 [0055] After selection of the desired accompaniment, 
the user may select a particular style of music that he 
wishes to play, but the default is that the computer 14 
chooses the style that has been pre-associated with 
each of the possible background tracks. Once the style 

55 is determined by either default or user selection, the 
computer 14 loads into memory 22 the data relevant to 
that style. 

[0056] The user is then allowed by the computer 14 
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to select an instrument from a list of a plurality of possi- 
ble instruments (step 70). In a preferred embodiment, 
the instrument list is stored by the computer 1 4 on, e.g., 
the hard drive 36 or the CD-ROM 40. For each instru- 
ment in the list, there are stored all kinds of data relevant 
to that instrument. These instrument-specific data are 
representative of, for example, the functionality of actu- 
ators (e.g., buttons) on the joystick 1 2 or other input de- 
vice 30, whether the instrument can play chords and 
what voicings for the chords, the timbre of the instrument 
which is the characteristic quality of sounds made by the 
instrument independent of pitch and loudness, pitch en- 
velopes for one or more notes that the instrument is ca- 
pable of producing, and the pitch range for the instru- 
ment. 

[0057]~ A more complete description of the setup stage 
is provided below with reference to FIG. 4B. In FIG. 4B, 
the user-selectable items include the skill level 72 (nov- 
ice through expert), the type of interface 74 (e.g., joy- 
stick, game pad, MIDI keyboard, spatial sensors, etc.), 
the type of instrument 76 (e.g., guitar, piano, saxophone, 
etc.), the background track 78 (i.e., the accompaniment 
piece over which the user wishes to play), and a musical 
style 80 in which the user wishes to play. Each back- 
ground track has associated with it a default musical 
style that is most compatible with the accompaniment, 
but the user may choose an alternative style for the sake 
of experimentation. Once the user makes a selection 
from the plurality of choices available for each of these 
user-selectable items (e.g., novice, expert, or some- 
where in between for the skill level 72 item), a whole set 
of data/variables associated with that selection for that 
item are loaded into memory 22 from the hard drive 36, 
and those data/variables are used to configure the sys- 
tem 1 0 in a particular way, as described hereinafter with 
reference to FIG. 4C. 

[0058] Referring nowto FIGS. 1 , 2, 3A, 3B, and 5, with 
the setup stage complete, the programmed computer 
14 waits until the user presses a "start" button (e.g., a 
graphic button on the monitor 26 which the user points 
to with the mouse 34 and clicks on). See step 82 in FIG. 
5. Once "start" has been indicated, the playback of the 
background track commences (step 84). In the pre- 
ferred embodiment, the user then uses the joystick 12 
(or other similarly simple-to-operate input device) to cre- 
ate music with the system 10. As described previously 
with reference to FIG. 3A, the user must depress and 
hold the play button 50 on the joystick 12 (step 86) to 
enable the "rhythm generator" (discussed hereinafter) 
and thus the system 10 (step 88). 
[0059] The following is a further description of the us- 
er-selected (or default) configuration data/variables 
(FIG. 4B), and the way that they affect the operation of 
the system 10 according to the invention. Referring to 
FIG. 4C, the configuration data associated with the se- 
lected skill level 72, interface type 74, instrument type 
76, and musical style 80 are provided to one or more of 
the functional blocks of the system 10 of the invention 



as depicted. These functional blocks are all described 
hereinafter with reference to FIG. 10. The selected 
background track 78 also is provided to some of the 
functional blocks. 

5 [0060] Still referring to FIG. 4C, some of the configu- 
ration data for the selected skill level 72 is provided to 
an automator functional block, and some is provided to 
an interface processor functional block. Both of those 
blocks are described hereinafter with reference to FIG. 

10 10. The automator receives data about how much sys- 
tem automation should be turned on. For a novice, full- 
automation will be turned on such that the novice user 
need only operate the play button to create music, for 
example. For each level higher than novice, the level of 

75 system automation decreases to the point where an ex- 
pert is given the most amount of control possible. For 
an expert, the system might enable all buttons and axes 
on the joystick and a plurality of additional buttons. 
These additional buttons typically are keys of the alpha- 

20 numeric computer keyboard (or a Ml Dl keyboard or sim- 
ilar device) . The interface processor is told what but- 
tons, sliders, etc. on the interface (e.g., joystick and/or 
keyboard) are enabled/disabled. 
[0061] Some of the configuration data for the selected 

25 interface type 74 is provided to a gesture analyzer func- 
tional block, and some is provided to the interface proc- 
essor. Both of those blocks are described hereinafter 
with reference to FIG. 10. The gesture analyzer can be 
a joystick-sensing system or possibly an electronic eye 

30 system, and the data it receives indicates the user's ges- 
tures or movements (with the joystick) for which the ges- 
ture analyzer should be looking and also the corre- 
sponding system functions that should be triggered as 
a result of those gestures. The interface processor is 

35 told what non-instrument-specific system functions 
should be triggered by each of the various enabled ac- 
tuators (e.g., buttons) on the interface (e.g., joystick). 
[0062] Some of the configuration data for the selected 
instrument type 76 is provided to the interface proces- 

^0 sor, and other data is provided to a chord builder, a tim- 
bre manager, an envelope manager, an articulator, and 
a pitch selector. All of these functional blocks are de- 
scribed hereinafter with reference to FIG. 10. The inter- 
face processor is told what instrument-specific system 

45 functions should be triggered by each of the various en- 
abled actuators (e.g. , buttons) on the joystick. The chord 
builder is told whether or not the selected instrument can 
play chords and if so what are the characteristic chord 
structures or voicings for the selected instrument. The 

50 timbre manager is provided with the timbre information 
for the selected instrument. The envelope manager is 
told the pitch envelopes to be used for the selected in- 
strument in order to shape the pitch of the note (e.g., 
bend it up or down) to simulate how that instrument 

55 would sound if played by a trained musician. The artic- 
ulator is told whether slurring the chosen instrument will 
affect the attack portion of the timbre for that instrument. 
The pitch selector is provided with information about the 
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range of pitches (lowest to highest) that the selected in- 
strument could produce if played by a trained musician. 
[0063] Some of the configuration data for the default 
(or selected) musical style 80 is provided to the pitch 
selector, and the other data is provided to a sustainer, 
a riffer, an accenter, and the rhythm generator. These 
functional blocks are described hereinafter with refer- 
ence to FIG. 10. The pitch selector is provided with in- 
formation about various melodic constraints for the giv- 
en style such as at which times (metrically) consonant 
notes are more likely. The sustainer is told which times 
(metrically) are eligible for sustaining notes in the given 
style. The riffer is provided with "riffs" (which generally 
are rhythm blocks coupled with melodic contours) ap- 
pfepriate for the given style, and these are used for ef- 
fects such as grace notes, gliassandi, trills, tremolos, 
and other melodic ornaments. The accenter and the 
rhythm generator are both provided with rhythm blocks 
associated with the given style. 
[0064] As somewhat of an aside, it is noted that each 
of the background tracks from which the user can select 
comprises: (i) a harmony track 90; (ii) a tempo track 92; 
and (iii) a MIDI and/or audio track 94. The third compo- 
nent of the background track typically is either a MIDI 
track or an audio track. In either case, it is a data file of 
the music over which the user wants to play a solo or 
melody. It could be a song by, for example, James 
Brown, Black Sabbath, or Barry Manilow. The other two 
tracks, the harmony and tempo tracks, are created from 
scratch by the system programmers/designers based 
on the song (i.e., the MIDI/audio track). Unlike the MIDI/ 
audio track, the harmony and tempo tracks are not re- 
cordings of a song that a person could listen to and rec- 
ognize. Instead, these two tracks contain data that the 
system 10 of the invention utilizes in selecting and play- 
ing notes (under the user's control) that are appropriate 
for the song. The harmony track contains key and chord 
information about the song. More specifically, it contains 
data representative of the key and chord at any partic- 
ular point in the song. The tempo track contains data 
representative of the timing of the song. It essentially 
provides timing information about the song in the form 
of a time grid. 

[0065] The harmony track provides to the pitch selec- 
tor the current "key" and the current "chord". The "key" 
data provided to the pitch selector includes both the root 
note of the key and the type of key. Examples are: "F 
major" (where "F" is the root and "major" is the type) 
which is defined by the notes F, G, A, B-flat, C, D, and 
E; "D minor" (where B D" is the root and "minor" is the 
type) which is defined by the notes D, E, F, G, A, B-flat, 
and C; and "C major" which includes the notes C, D, E, 
F, G, A, and B. 

[0066] The tempo track provides to the pitch selector 
the aforementioned time grid which the pitch selector 
uses to select a pitch from one of two or more classes 
of pitches. The pitch selector makes this selection be- 
tween or among classes based, in part, on the current 



metric position. For instance, the two classes might be 
chord tones (i.e. , notes in the current chord) and passing 
tones (i.e., notes in the current key or scale). For exam- 
ple, it is a general melodic principle that chord tones 

5 should normally be played on the beat (e.g., the down 
beat or other strong beats) and passing tones should 
normally be played off the beat or on weak beats. Given 
the current metric position with respect to the beat or 
measure, the pitch selector will select the most appro- 

10 priate pitch class. Then, a particular pitch from that class 
is selected by the pitch selector based on the current 
harmony and the current pitch-related joystick position. 
An example is when the current chord is a C chord and 
the current key is "D minor" in which case a G note might 

is be played on a strong beat and a B-flat note might be 
played off the beat or on a weak beat. It is noted that 
some notes may, and very often will, overlap between 
or among the plurality of classes such as in the previous 
example where the current chord is C (i.e., the chord 

20 tones are C, E, and G) and the key is "D minor" (i.e., the 
passing tones are D, E, F, G, A, B-flat, and C). 
[0067] The tempo track also provides data to the 
rhythm generator. The rhythm generator gets the afore- 
mentioned time grid which the rhythm generator uses to 

25 synchronize the user-created melody or solo line with 
the background track. 

RHYTHM BLOCKS: 

30 [0068] The "rhythm blocks" alluded to above are now 
described in detail. Rhythm blocks are fundamental to 
the operation of the invention. Rhythm blocks are uti- 
lized by the "rhythm generator" (described hereinafter) 
to produce rhythmic signals when, for example, the user 
35 depresses the play button 50 on the joystick (FIGS. 3A 
and 5). As alluded to above with reference to FIG. 3B, 
rhythm blocks are organized by the system designer/ 
programmer into a plurality of groupings where each 
grouping ranges from a block with a lowest rhythmic ac- 
40 tivity for that group to a block with a highest rhythmic 
activity for that group. Once the musical style is selected 
(by the user or by default), the associated group or list 
of rhythm blocks are copied into the memory 22 of the 
computer 1 4 from, for example, the hard drive 36. A giv- 
45 en style of music might cause a set of rhythm blocks to 
be copied into memory that range from a whole note at 
the lowest activity level block to a sixty-forth note at the 
highest activity level block. In such a case, and if the 
joystick of FIG. 3B is used as the interface, pulling the 
so handle of the joystick 1 2 all the way backward and hold- 
ing it there would result in a series of whole notes to be 
output by the rhythm generator and played by the sys- 
tem, holding the handle all the way forward would cause 
a series of sixty-forth notes to be output, and moving the 
55 handle to a position somewhere therebetween would re- 
sult in the output of a series of notes having a rhythmic 
activity level somewhere between whole notes and six- 
ty-forth notes such as eighth notes. Also, and perhaps 
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more importantly, as the user moves the handle back 
and forth, the rhythmic output is varied accordingly and 
the user is thus able to, for example, follow a half note 
with a sixteenth note, and the user generally is able to 
create a rhythmic output of any variety or combination. 
[0069] A rhythm block can be thought of as a data 
structure that has five fields: (i) identifier (a name and 
an identification number); (ii) length; (iii) event list; (iv) 
dynamics list; and (v) durations list. A rhythm block does 
not need to have a value in each of these five fields, but 
every rhythm block typically will have at least an identi- 
fier, a length, and an event list. In the current embodi- 
ment, all five fields are used. The name component of 
the identifier indicates the type of note(s) in the rhythm 
block. The length of a rhythm block typically is one beat, 
buTtri general the length can be more than one beat such 
as 1.5 beats or two beats. The system designer/pro- 
grammer has set one beat to equal 480 "ticks" of a 
scheduler. The preferred scheduler is OMS 2.0 which is 
available from Opcode Systems of Palo Alto, CA, al- 
though another scheduler could be used such as Ap- 
ple's QuickTime product. The event list specifies the 
precise times (in units of ticks) within a beat when the 
rhythm is to play. In the disclosed embodiment, the dy- 
namics (i.e., volume, loudness, accent - which is called 
"velocity" in MIDI terminology) are measured or speci- 
fied on a scale from 0 to 127 where 0 is silent and 127 
is maximum velocity. The dynamics list specifies the 
loudness of each of the notes in the rhythm block. The 
duration list of the rhythm block sets how long the note 
(s) should last in units of ticks. 

[0070] Referring to FIG. 6A, one possible rhythm 
block defines two eighth notes. The block has a length 
of one beat and an event list with the values 0 and 240 
which means that the first eighth note will sound at the 
beginning of the beat and the second one will sound at 
exactly half way through the beat (240/480 = 1/2). The 
dynamics list has values of 84 and 84, implying mezzo 
forte loudness for each note. The durations list has val- 
ues of 240 and 240, implying legato articulation for each 
eighth note. In other words, the first eighth note will last 
until the second one plays (i.e., for ticks 0 through 239), 
and the second eighth note will last until the end of the 
beat (i.e., for ticks 240 to 479). The repeat notation in 
the "musical equivalent" section of this example indi- 
cates that the rhythm generator will continue to output 
this same rhythm block unless the user moves the po- 
sition of the handle of the joystick 12. The same is true 
for all rhythm blocks; once the user has depressed the 
play button 50 on the joystick, the only way the rhythm 
generator will stop outputting the appropriate rhythm 
blocks is if the play button 50 is released. 
[0071] Referring to FIG. 6B, another example of a 
rhythm block is two syncopated 16th notes. This block 
has a length of one beat, and an event list with the values 
120 and 360 which means that the first sixteenth note 
will sound one-quarter of the way through the beat 
(120/480 = 1/4) and the second sixteenth note will play 



at three-quarters of the way through the beat (360/480 
= 3/4). The dynamics are as in the previous example. 
The durations list has values of 120 and 120, implying 
detached articulation. 

5 [0072] Referring to FIG. 6C, a third example of a 
rhythm block is a dotted eighth note cross-rhythm. In this 
example, the length is not one beat but instead 1 .5 beats 
(i.e., 720 ticks). The event list has the value zero which 
means that the dotted eighth note will play at the begin- 

70 ning of the block. The dynamics and duration are as in- 
dicated in the figure. 

[0073] Referring to FIG. 6D, the final example shows 
two eighth notes with an offbeat accent. The length is 
one beat or 480 ticks, and the event list values of 0 and 

75 240 will cause the first eighth note to play at the begin- 
ning of the beat and the second one to play in the middle 
of the beat, as in FIG. 6A. The dynamic values of 72 and 
96 will cause the second note to sound accented. The 
duration values of 120 and 240 will further distinguish 

20 the two notes. 

[0074] Once the system designer/programmer has 
defined all desired rhythm blocks, he assembles a plu- 
rality of groups or lists using the rhythm blocks as the 
items in the list. As described previously, each grouping 

25 contains two or more rhythm blocks organized in order 
of increasing rhythmic activity. The rhythm blocks and 
the groupings of them are essentially transparent to the 
user. The musical style that is selected by the user or 
by default (FIGS. 4A-4C) determines the group(s) of 

30 rhythm blocks that will be available to the user. 

[0075] Referring to FIG. 7A, one example of a style 
and its associated rhythm block data is the slow rock 
musical style. Associated with this style are four sepa- 
rate groupings of rhythm blocks, each one having its 

35 rhythm blocks ordered in increasing rhythmic activity. 
The four groupings of rhythm blocks are titled "Normal", 
"Syncopated", "Alternate 1 ", and "Alternate 2". A user 
can be allowed to switch among these four groups by, 
for example, operating a button on his joystick. Like the 

40 example in FIG. 3B, in this example, with the handle of 
the joystick in the leftmost position, the rhythm block at 
the top of the appropriate list is selected, and with the 
handle in the rightmost position, the rhythm block at the 
bottom of the appropriate list is selected. This example 

45 of a musical style shows other data or variables that can 
be determined by the style configuration 80 (FIGS. 4B 
and 4C), and these are "swing" and "half-shuffle" pa- 
rameters. In the slow rock style example, the swing is 
set to 0% and the half-shuffle also is set to 0%. Swing 

50 and half-shuffle are defined below. 

[0076] The "swing" parameter is a measure of how 
much the offbeat (or upbeat) eighth note should be de- 
layed. The delay range is 0 to 80 ticks where 0% corre- 
sponds to 0 ticks and 100% corresponds to 80 ticks. 

55 Thus, a swing of 50% means to delay the offbeat eighth 
notes by 40 ticks. Swing is a well-known term used by 
musicians and composers to indicate the offbeat eighth 
note delay described above. 
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[0077] The "half-shuffle" parameter is a measure of 
how much the upbeat sixteenth notes (occurring at ticks 
120 and 360 within the beat) should be delayed. The 
delay range is 0 to 40 ticks where 0% corresponds to 0 
ticks and 100% corresponds to 40 ticks. Thus, a half- 
shuffle of 50% means to delay the offbeat sixteenth 
notes by 20 ticks. Half -shuffle is a well-known term used 
by musicians and composers to indicate the upbeat six- 
teenth note delay described above. 
[0078] Referring to FIG. 7B, another example of a 
style and its associated rhythm block data is the fast 
blues musical style. Associated with this style are three 
separate groupings of rhythm blocks, each one having 
its rhythm blocks ordered in increasing rhythmic activity. 
Ibe-tbree groupings of rhythm blocks are titled "Nor- 
mal+Syncopated", "Alternate 1 ", and "Alternate 2". A us- 
er can be allowed to switch among these three groups 
by, for example, operating a button on his joystick. Like 
the style example in FIG. 7A, in this example, with the 
handle of the joystick in the leftmost position, the rhythm 
block at the top of one of the three lists is selected, and 
with the handle in the rightmost position, the rhythm 
block at the bottom is selected. The swing parameter for 
this example style is set to 50% which means that all 
offbeat eighth notes will be delayed by 40 ticks. As with 
the previous style example, the half-shuffle parameter 
is set to 0% which means no delay of the offbeat six- 
teenth notes. 

RHYTHM GENERATOR: 

[0079] The "rhythm generator" that outputs the 
above-described rhythm blocks is now described in de- 
tail. The rhythm generator allows the user to produce 
"musically correct" rhythms without requiring the user to 
have the physical dexterity needed to play those 
rhythms on a traditional or known instrument. The user 
can enable and disable the rhythm generator with the 
play button on the joystick. This button causes the music 
to start and stop, and thus it can be used by the user to 
simulate the way an improvising musician starts and 
stops musical phrases during a solo. The user can use 
a combination of buttons and continuous controllers (e. 
g., the axes of a joystick handle, faders, sliders, etc.) on 
his interface to control the activity and complexity of the 
generated rhythms. 

[0080] Referring to FIGS. 8 and 10, the rhythm gen- 
erator 100 selects a rhythm block (from the group of 
rhythm blocks provided by the style configuration 80, 
FIGS. 4B and 4C) in response to every rhythm-related 
input signal from the joystick or other similarly simple- 
to-operate interface 12 (steps 202 and 204). Once the 
rhythm generator 100 selects a rhythm block, it trans- 
mits messages to a note builder functional block 102, 
the riffer 104, and the accenter 106. 
[0081] To the note builder 102, the rhythm generator 
100 sends a "play note" instruction at the correct times 
as defined by the rhythm block itself (step 206). A "play 



note" instruction includes all of the information defined 
by the rhythm block, specifically the name of the block, 
its length, and its event list as well as either specified or 
default dynamics and duration information. 

5 [0082] If the rhythmic activity is sufficiently high, it can 
be difficult or impossible for the user to manipulate the 
input device (e.g., joystick) fast enough to avoid rapid 
repetition of the same pitch. To remedy this situation, 
when the rhythmic activity gets sufficiently high, the 

to rhythm generator 1 00 sends an instruction to enable the 
riffer 104. Once enabled, the riffer 104 disables the 
rhythm generator 100, and the riffer 104 then automat- 
ically outputs pre-stored melodic elaborations (e.g., ar- 
peggios). When the rhythmic activity becomes suffi- 

is ciently low again, the riffer 104 will return control to the 
rhythm generator 100. The riffer 104 is described in 
more detail hereinafter under the "riffer" heading. 
[0083] The information transmitted by the rhythm gen- 
erator 100 to the accenter 106 is the identification 

20 number for the current rhythm block. The accenter 106 
uses that ID number to add accent patterns, as de- 
scribed hereinafter under the accenter heading. 

PITCH SELECTOR: 

25 

[0084] Referring to FIGS. 9 and 10, the "pitch selec- 
tor" 108 ensures that the pitches of the notes generated 
by the user are "musically correct". In response to every 
pitch-related input signal from the joystick or other sim- 

30 jiarly simple-to-operate interface 12, the pitch selector 
108 selects a pitch for playback (steps 208 and 210). 
The pitch selector selects an appropriate pitch as a func- 
tion of the pitch-related input signals from the joystick, 
the current key and chord of the accompaniment (pro- 

35 vided by the harmony track 90 part of the background 
track 78, FIG. 4C), the current metric position (provided 
by the tempo track 92 part of the background track 78), 
and information about previous pitches played. See 
steps 218, 210, 208, 212, 216, and 21 4 of FIG. 9. Note 

40 that the metric position is an indication of the current 
position in, for example, a beat (e.g., on the beat or off 
the beat) or a measure (e.g., strong beat or weak beat), 
and it generally is independent of the harmony associ- 
ated with that same point in time. Once a pitch has been 

45 selected, the pitch selector sends the selected pitch to 
the note builder 102 to be used in the next note that is 
played (step 220). 

[0085] As described previously, the pitch selector 1 08 
selects an appropriate pitch from one of a plurality of 

so classes of pitches. The pitch selector 108 makes this 
selection between or among classes based on the fac- 
tors disclosed in the preceding paragraph. As an exam- 
ple, there might be two classes where one is a collection 
of chord tones (i.e., notes in the current chord), another 

55 is a collection of passing tones (i.e. , notes in the current 
key or scale), and another is a collection of chromatic 
tones. 

[0086] For example, a general melodic principle is 
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that chord tones should normally be played on the beat 
(e.g., the down beat or other strong beats) and passing 
tones should normally be played off the beat or on weak 
beats. Given the current metric position with respect to 
the beat or measure, the pitch selector will select the s 
most appropriate pitch class. Then, a particular pitch 
from that class is selected by the pitch selector based 
on the current harmony and the current pitch-related joy- 
stick position. An example is when the current chord is 
a C chord and the current key is "D minor" in which case 10 
a G note might be played on a strong beat and a B-flat 
note might be played off the beat or on a weak beat. It 
is noted that some notes may, and very often will, over- 
lap between or among the plurality of classes such as 
in the previous example where the current chord is C (i. is 
e7,Tie"chord tones are C, E, and G) and the key is "D 
minor" (i.e., the passing tones are D, E, F, G, A, B-flat, 
and C). 

[0087] When selecting a pitch class, the pitch selector 
also utilizes historical information about the melody The 20 
pitch selector utilizes information such as the pitch 
classes of the preceding notes, the actual pitches of the 
preceding notes, and other melodic features of the pre- 
ceding notes such as melodic direction. For example, a 
general melodic principle is that if a melody leaps to a 2$ 
non-chord tone, the melody should then step in the op- 
posite direction to the nearest chord tone. 
[0088] Once pitch class is determined by the pitch se- 
lector 108, the pitch selector 108 then utilizes the pitch- 
related input signals to select a particular pitch from 30 
within that class. In general, the pitch-related input sig- 
nal corresponds directly to either: (i) pitch register (i.e., 
how high or low the pitch of the note should be); or (ii) 
change in pitch register (i.e., whether the next pitch 
should be higher or lower than the preceding pitch and 3$ 
by how much). 

INTERFACE PROCESSOR: 

[0089] Referring to FIG. 10, the "interface processor" 40 
functional block 110 is responsible for channeling or 
"mapping" the signals from the input device 1 2 (e.g. , joy- 
stick) to the correct system functional blocks. There are 
many ways that the interface processor 110 could have 
been configured. In the disclosed embodiment, the in- *s 
terface processor 1 1 0 is configured to transmit messag- 
es to the rhythm generator 100, the pitch selector 108, 
the sustainer 1 1 2, the riffer 1 04, a licker 1 1 4, the timbre 
manager 116, the envelope manager 118, the chord 
builder 120, an articulator 122, and the accenter 106. so 
[0090] To the rhythm generator 100, the interface 
processor 110 sends the position of the play button 50 
on the joystick 12 which enables/disables the rhythm 
generator 100. Also sent is the position of the joystick 
handle along the forward/back axis, or whatever axis is ss 
used to increase/decrease rhythmic activity. The inter- 
face processor 110 also sends to the rhythm generator 
100 the position of the other buttons on the joystick 



which can be used to change rhythm blocks for rhythmic 
special effects such as cross-rhythms, poly-rhythms, 
and syncopation. 

[0091] To the pitch selector 108, the interface proces- 
sor 110 sends the position of the joystick's handle along 
the left-right axis, or whatever axis is used to raise/lower 
the pitch of the notes. 

[0092] To the sustainer 112, the interface processor 
110 sends the position of the sustain button 52 on the 
joystick 12 which enables/disables the sustainer 112. 
[0093] To the riffer 104, the interface processor 110 
sends the position of the various riff buttons which ena- 
ble/disable the riffer's functions, and it sends information 
about the release of the sustain button 52 and the si- 
multaneous position of the joystick handle along the left- 
right axis to trigger the riffer 104. 
[0094] To the licker 114, the interface processor 110 
sends the position of the lick repeat button 54 which en- 
ables/disables the licker 114, and it sends information 
about when the lick repeat button 54 is held depressed 
and the coincident position of the joystick handle along 
the left/right axis to move the lick up and down in register 
on each repeat. 

[0095] To the timbre manager 1 1 6, the interface proc- 
essor 110 sends the position of the various timbre but- 
tons which enable/disable various functions of the tim- 
bre manager 116, and it sends information about when 
the sustain button 52 is held depressed and the coinci- 
dent position of the joystick handle along the forward/ 
backward axis to control continuous blending of multiple 
timbres. 

[0096] To the envelope manager 118, the interface 
processor 110 sends the position of the various enve- 
lope buttons which enable/disable various functions of 
the envelope manager 118, and it sends information 
about when the sustain button 52 is held depressed and 
the coincident position of the joystick handle along the 
left/right axis to control pitch bending. 
[0097] To the chord builder 1 20, the interface proces- 
sor 110 sends the position of various chord buttons 
which enable/disable various functions of the chord 
builder 120. 

[0098] To the articulator 122, the interface processor 
110 sends the position of various articulation buttons 
which enable/disable various functions of the articulator 
122. 

[0099] To the accenter 106, the interface processor 
110 sends the position of various accenter buttons 
which enable/disable various functions of the accenter 
106. 

[0100] The input device 1 2 typically will not include all 
of these buttons, although it may FIGS. 3A and 3B show 
only three buttons, but there can be a variety of other 
buttons provided on, for example, the base of the joy- 
stick (or they can be the keys of the computer keyboard 
or the keys of a MIDI keyboard). 
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GESTURE ANALYZER: 

[0101] Referring to still FIG. 10, instead of allowing 
the user only to provide input to the system by pressing 
a button or moving a continuous controller on the input 
device 12 (e.g., joystick), the gesture analyzer 124 can 
be used to allow the user to trigger specific system func- 
tions with "gestures". For example, the system can be 
configured to recognize "wiggling the joystick wildly" as 
a trigger for some special rhythmic effect. The gesture 
analyzer 1 24 is responsible for analyzing the user's ma- 
nipulation of the interface and determining whether or 
not the user is, for example, currently "wiggling the joy- 
stick wildly" which would mean the gesture analyzer 1 24 
should send the appropriate signal to the interface proc- 
essor 1 1 0 in order to enable the desired rhythmic effect. 

SUSTAINER: 

[0102] The sustainer 112 allows the user to sustain a 
played note for an indefinite duration. When the sustain- 
er 112 is enabled, it sends an instruction to the rhythm 
generator 100. This instruction tells the rhythm genera- 
tor 100 to interrupt its normal stream of "play note" mes- 
sages and sustain the next played note until further no- 
tice. When the sustainer 112 is disabled, it sends an in- 
struction to the rhythm generator 100 to silence the sus- 
taining note and then resume normal note generation. 

RIFFER: 

[0103] The riffer 104 is used to play back "riffs" which 
are pre-stored data structures that each contain: (i) a 
time-indexed list of "play note" events; and (ii) a list 
specifying a melodic direction offset (up or down, and 
by how much) for each of those "play note" events. This 
data structure enables the riffer 104 to automatically 
perform musical "riffs" for the purpose of melodic auto- 
mation. Some examples of pre-stored riffs are: grace 
notes, mordents, trills, tremolos, and glissandi. Another 
use for riffs is to add melodic contours (e.g., arpeggios) 
when the rhythmic activity gets so high that it would be 
difficult for the user to add plausible melodic contours 
manually. When enabled, the riffer 104 transmits mes- 
sages' to the rhythm generator 100 and the note builder 
102. 

[0104] To the rhythm generator 100, the riffer 104 
sends an instruction to stop generating rhythms when 
the rhythmic information for the note builder 102 starts 
being supplied by the riffer 104. 
[0105] To the note builder 102, the riffer 104 sends an 
instruction to play a note (or chord) at the correct times 
as determined by the current riff. This "play note" in- 
struction is also accompanied by a melodic offset, du- 
ration, and loudness (i.e., MIDI "velocity") as specified 
by the current rhythm block. 



LICKER: 

[0106] The licker 114 allows the user to "capture" 
pleasing melodic fragments from the immediate past 
5 and replay them in rapid succession. Licks are stored in 
the same data structure format as riffs. However, licks 
are not pre-stored. The user's solo or melody is record- 
ed automatically in the memory 22 of the computer 14 
in real-time as it is created by the user. When the licker 
to 114 is enabled (by the lick repeat button 54), it chooses 
a lick of random length from recent memory (usually a 
few beats or less) and saves the lick into the riff data 
format. The licker 14 then passes that lick to the riffer 
104 along with an instruction to enable the riffer 104. 
*s The licker 114 then resumes recording the generated 
music. 

TIMBRE MANAGER: 

[0107] This functional block, the timbre manager 116, 
allows the user to affect the timbre of the current solo 
instrument. This is accomplished by sending the gener- 
ated notes to multiple MIDI channels, each of which is 
using a different MIDI patch (timbre). The timbre man- 
ager 116 can then continually adjust the MIDI volume of 
these respective MIDI channels, thus changing the tim- 
bral "mix" of the output. Note that some MIDI tine gen- 
erators also allow direct manipulation of timbre by con- 
trolling synthesis parameters. The default MIDI patches 
for each instrument are provided in the instrument con- 
figuration 76. 

ENVELOPE MANAGER: 

[0108] The envelope manager 118 allows the user to 
modulate the pitch and loudness of sounding notes to 
achieve multiple effects such as pitch bends or crescen- 
di. When enabled, the envelope manager 118 uses pitch 
bend and loudness (i.e., MIDI "velocity") envelopes to 
alter the playback of notes. These envelopes are either 
pre-stored (in which case they are provided in the instru- 
ment configuration 76) or controlled in real-time by sig- 
nals from the input device 12. The envelope manager 
118 also automatically adds minute random fluctuations 
in pitch to some instruments (specifically string and wind 
instruments) so as to mimic human performance imper- 
fections. 

CHORD BUILDER: 

[0109] The chord builder 120 directs the note builder 
102 when to perform chords instead of single notes. 
When enabled, the chord builder 1 20 sends a message 
to the note builder 102 telling it: (i) how many chord 
notes to play in addition to the main melody note just 
created; (ii) how close together (in pitch) those chord 
notes should be; and (iii) whether those chord notes 
should be above or below the main melody note. This 
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information is provided to the chord builder in the instru- 
ment configuration 76. 

ARTICULATOR: 

[0110] The articulator 122 allows the user to add ar- 
ticulation effects to the generated notes. Articulation is 
defined as the way in which individual notes are at- 
tacked and how much rest space is left between se- 
quential notes. For example, if the "staccato" function of 
the articulator is enabled, it will send an instruction to 
the note builder 102 to shorten the duration of the next 
generated note. If the "slur" function of the articulator is 
enabled, it will send an instruction to the note builder 
1 02 to lengthen the duration of the next generated note, 
ancTit will tell the note builder 102 to enable MIDI Porta 
Mode (with Porta Time = 0), in which case the attack 
portion of the timbre envelopes of new notes will not be 
re-articulated. This is analogous to slurring notes on a 
traditional instrument. 

ACCENTER: 

[0111] The accenter 1 06 allows the user to add accent 
patterns to the generated notes. The accenter 106 has 
knowledge (from the style configuration 80) of all of the 
available rhythm blocks. The accenter 106 also has 
knowledge (from the rhythm generator 100) of which of 
those rhythm blocks are currently being used. When en- 
abled, the accenter 106 uses this information to choose 
a complimentary rhythm block for use as an accenting 
template or an "accent block" in which a certain note or 
notes have a higher loudness value than the loudness 
of the corresponding note(s) in the rhythm block from 
the rhythm generator 1 00. At the times of the "play note" 
events defined by that accent block, the accenter sends 
messages to the note builder 102 instructing it to add a 
specified accent to any notes generated at that time. 

NOTE BUILDER: 

[0112] The note builder 102 combines all of the per- 
formance information from all of the other enabled func- 
tional blocks. As an example, the note builder 102 can 
integrate a "play note" instruction from the rhythm gen- 
erator 100, a pitch from the pitch selector 108, a timbre 
adjustment from the timbre manager 116, a pitch bend 
value from the envelope manager 118, a duration value 
from the articulator 122, and a loudness (i.e., MIDI "ve- 
locity") value from the accenter 106. These data come 
into the note builder 1 02 and are integrated thereby and 
then sent out to the MIDI output device 16. Other outputs 
from the note builder 102 go to the pitch selector 108 
and the licker 114. 

[0113] When instructed by the chord builder 120 to 
play a chord, the note builder 102 causes the pitch se- 
lector 108 to execute X number of additional times in 
order to produce X number of pitches until the desired 



chord has been constructed, where X and the desired 
chord are determined the chordal parameters supplied 
by the chord builder 120. 

[0114] To the memory buffer of the licker 1 1 4, the note 
s builder 102 sends all of the note builder's output such 
that the licker 1 1 4 will always have a record of what has 
been played and will be able to perform its function 
(which is described hereinabove) when called upon by 
the user to do so. 

10 

MIDI OUTPUT DEVICE: 

[0115] This block 16 is the actual sound generating 
hardware (or, in some cases, software as mentioned 
is previously) that "renders" the MIDI output stream from 
the note builder 102 meaning it translates the MIDI out- 
put stream into an audio signal which may then be am- 
plified and broadcast. 



[0116] The MIDI output stream from the note builder 
102 also can be recorded. For example, the MIDI output 
stream can be sent to the hard drive 36 of the computer 
25 14 and stored thereon. This allows a user to save his 
performance and easily access (e.g., listen to) it at any 
time in the future. 

AUTOMATOR: 

30 

[0117] The system of the invention thus clearly pro- 
vides the user with a large number of control function- 
alities. Given enough buttons and faders, a user could 
independently control rhythm, pitch, sustain, riffs and 

35 licks, timbre, pitch envelopes, chords, articulation, and 
accents. However, such a great degree of control would 
be overwhelming for most users. 
[0118] The purpose of the automator 1 30 is to act like 
a user's assistant and to automatically control many of 

40 these system functions thereby allowing the user to con- 
centrate on just a few of them. The automator 1 30 is told 
which system functions to control by the skill level con- 
figuration 72. 

[0119] In FIG. 10, the automator 130 is a different 
45 shape than all of the other blocks to indicate that it re- 
ceives information from every block in FIG. 10 (even 
though all of the lines are not shown). The automator 
1 30 has access to all of the information in the entire sys- 
tem, and it uses this information to decide when to en- 
50 able various system functions. 

[0120] As an example, the automator 130 can regu- 
larly or occasionally send pre-stored pitch-related input 
signals to the pitch selector 1 08. This might be done, for 
example, if the user has identified himself as having a 
55 very low skill level (i.e., a beginner) to the skill level con- 
figuration 72. ' 

[0121] As another example, the automator 130 can 
regularly or occasionally send pre-stored rhythm-relat- 
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ed input signals to the rhythm generator 100. Again, this 
might be done, for example, if the user has identified 
himself as having a very low skill level (i.e., a beginner) 
to the skill level configuration 72. 
[0122] Another example is where the automator ran- 
domly or algorithmically enables one or more functional 
blocks (e.g., the timbre manager 116, the envelope 
manager 1 1 8, the chord builder 1 20, the articulator 1 22, 
the accenter 106, and/or the riffer 104) in order to add 
automatically complexity to the user's solo line. 
[0123] One component of this complexity is instru- 
ment-specific performance parameters such as pitch 
bends and timbre substitutions (e.g., guitar harmonics). 
Another component of this complexity is automatic or- 
namentation of the score by the addition of effects such 
as grace notes, tremolos, glissandi, mordents, etc. 
[0124] In general, the automator is an electronic sys- 
tem for processing a musical score to modify automati- 
cally the score by adding instrument-specific perform- 
ance parameters or musical ornamentation. The musi- 
cal score is represented by digital data such as MIDI 
data. The score can be the score that is created in real- 
time by the system according to the invention, or it can 
be a score which has been created in the past and 
stored or recorded on, for example, a computer hard 
disk drive or other computer-readable data storage me- 
dium. 

[0125] Variations, modifications, and other implemen- 
tations of what is described herein will occur to those of 
ordinary skill in the art without departing from the scope 
of the invention as claimed. Accordingly, the invention 
is to be defined not by the preceding illustrative descrip- 
tion but instead by the following claims. 



Claims 

1 . A system for creating music in real time, comprising: 

an input mechanism (30) which provides 
rhythm -related input signals and pitch-related 
input signals; 

one or more computer storage media 
(22,36,38) having a plurality of selectable mu- 
sical accompaniment tracks over which music 
can be created and played in real time, having 
a plurality of rhythm blocks wherein each 
rhythm block defines, for at least one note, at 
least a time at which the note should be played, 
and for storing at least a portion of the music 
created in real time over a predetermined time 
interval in the past; 

a rhythm generator (100) which in real time: re- 
ceives the rhythm-related input signals from the 
input mechanism; selects one of the rhythm 
blocks (204) from the computer storage media 
based on the rhythm-related input signals; and 
outputs an instruction to play the note at the 



time defined by the selected rhythm block; 
a pitch selector (108) which in real time: re- 
ceives the pitch-related input signals from the 
input mechanism; selects an appropriate pitch 
5 based on the pitch-related input signals, a se- 

lected one of the musical accompaniment 
tracks, and the stored music; and outputs the 
appropriate pitch; and 

a sound generator (102) which in real time: re- 
io ceives instructions from the rhythm generator, 

pitches from the pitch selector, and the selected 
musical accompaniment track; and generates 
an audio signal representative of at least the 
created music. 

75 

2. The system of claim 1 wherein the input mechanism 
(30) comprises a joystick (12) having a base and a 
movable handle which a user manipulates to gen- 
erate the rhythm-related and pitch-related input sig- 

20 nals. 

3. The system of claim 2 wherein the joystick (1 2) gen- 
erates the rhythm-related input signals in response 
to the user's manipulations of the handle along a 

25 first predetermined axis (42,44), and the joystick 
generates the pitch- related input signals in re- 
sponse to the user's manipulations of the handle 
along a second predetermined axis (46,48). 

30 4. The system of claim 1 : 

wherein the input mechanism (30) also pro- 
vides musical effect-related input signals; 
further comprising a musical effect generator 

35 which receives the musical effect-related input 

signals from the input mechanism and which al- 
ters the instructions from the rhythm generator 
(100) or the pitches from the pitch selector 
(108) based on the musical effect-related input 

40 signals; and 

wherein the sound generator (102) receives the 
altered instructions and pitches and generates 
the audio signal which is representative of the 
at least the created music with musical effects. 

45 

5. The system of claim 4 wherein the input mechanism 
(30) comprises a joystick (12) having a base, a mov- 
able handle, and at least one button (50,52,54). 

so 6. The system of claim 5 wherein the joystick (1 2) gen- 
erates the rhythm-related input signals in response 
to a user's manipulations of the handle along a first 
predetermined axis (42,44), the joystick generates 
the pitch-related input signals in response to the us- 

55 er's manipulations of the handle along a second 
predetermined axis (46,48), and the joystick gener- 
ates the musical effect-related input signals in re- 
sponse to the user's manipulations of the button 
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(50,52,54) . 

7. The system of claim 1 wherein the rhythm generator 
(100) outputs the instruction to play the note at the 
time defined by the selected rhythm block and for a 
default duration and at a default loudness. 

8. The system of claim 1 wherein: 

at least one of the plurality of rhythm blocks in 
the one or more computer storage media 
(22,36,38) further defines, for at least one note, 
how long the note should last when played and 
how loud the note should be played; and 

the instruction output by the rhythm generator 

~~ (100) indicates the time at which the note 
should be played and the note's length and 
loudness as defined by the rhythm block. 

9. The system of claim 1 further including at least one 
speaker (18) for broadcasting the audio signal. 

10. The system of claim 9 further comprising an ampli- 
fier (18) for amplifying the audio signal before it is 
provided to the speaker for broadcast. 

11. The system of claim 10 wherein a programmed 
computer (20) performs the functions of the rhythm 
generator (100) and the pitch selector (108), and 
the amplifier and the speaker (18) are internal to the 
computer. 

12. The system of claim 11 wherein the programmed 
computer (20) also performs the functions of the 
sound generator (102). 

13. The system of claim 1 wherein the sound generator 
(102) comprises a MIDI tone generator. 

14. The system of claim 1 wherein a programmed com- 
puter (20) performs the functions of the rhythm gen- 
erator (100) and the pitch selector (108). 

15. The system of claim 14 wherein the programmed 
computer (20) also performs the functions of the 
sound generator. 

1 6. The system of claim 1 wherein the one or more com- 
puter storage media (22,36,38) comprise computer 
memory (22). 

17. The system of claim 1 wherein the one or more com- 
puter storage media (22,36,38) comprise computer 
memory (22) and a computer hard disk drive (36), 
the hard disk drive having the plurality of selectable 
musical accompaniment tracks over which music 
can be created and played. 



18. The system of claim 1 wherein the one or more com- 
puter storage media (22,36,38) comprise computer 
memory (22) and a CD-ROM having the plurality of 
selectable musical accompaniment tracks over 

s which the music can be created and played. 

19. A system for creating music as claimed in claim 1 , 
wherein the input mechanism (30) includes a device 
having at least a first axis of manipulation (42,44) 

10 and a second axis of manipulation (46,48), the input 
device providing rhythm-related signals in response 
to a user's manipulation of the input device with re- 
spect to the first axis and providing pitch-related sig- 
nals in response to the user's manipulation of the 
is input device with respect to the second axis. 

20. The system of claim 19 wherein the input device 
(30) comprises a joystick (12) having a movable 
handle which the user manipulates along the first 

20 axis (42,44) to cause the joystick to provide the 
rhythm-related signals and along the second axis 
(46,48) to cause the joystick to provide the pitch- 
related signals. 

25 21. The system of claim 19 wherein the input device 
(30) has the first and second axes arranged perpen- 
dicular to each other. 

22. The system of claim 19 wherein the input device 
30 (30) comprises a continuous controller. 

23. The system of claim 19 wherein the input device 
(30) comprises a mouse. 

35 24. The system of claim 19 wherein the input device 
(30) comprises a tracball. 

25. The system of claim 19 wherein the input device 
(30) comprises a fader. 

40 

26. The system of claim 19 wherein the input device 
(30) comprises a slider. 

27. The system of claim 19 wherein the input device 
45 (30) comprises a spatial sensor. 

28. The system of claim 19 wherein the input device 
(30) comprises a discrete controller. 

so 29. The system of claim 19 wherein the input device 
(30) comprises a game pad. 

30. The system of claim 19 wherein the input device 
(30) comprises a plurality of switches. 

55 

31. The system of claim 19 wherein the input device 
(30) comprises a plurality of buttons. 
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32. A system for creating music in real time as claimed 
in claim 1, wherein: 

the rhythm-related signals received are provid- 
ed in response to a user's manipulation of an 
input device (30) with respect to a first axis of 
manipulation (42,44) of the input device; 
the pitch-related signals received are provided 
in response to a user's manipulation of the input 
device with respect to a second axis of manip- 
ulation (46,48) of the input device. 

33. The system of claim 32 wherein: 

~— - — the rhythm-related signals received are provid- 
ed in response to the user manipulating the in- 
put device (30) along the first axis (42,44) which 
is perpendicular to the second axis (46,48); and 
the pitch-related signals received are provided 
in response to the user manipulating the input 
device along the second axis which is perpen- 
dicular to the first axis. 

34. The system of claim 33 wherein: 

the rhythm-related signals received are provid- 
ed in response to the user manipulating a han- 
dle of a joystick (12) along the first axis (42,44); 
and 

the pitch-related signals received are provided 
in response to the user manipulating the handle 
of the joystick along the second axis (46,48). 

Patentanspruche 

1. System zum Erzeugen von Musik in Echtzeit, mit: 

einem Eingabemechanismus (30), der rhyth- 
musbezogene Eingangssignale undtonhohen- 
bezogene Eingangssignale schafft; 
einem Oder mehreren Computerspeichermedi- 
en (22, 36, 38), die mehrere wahlbare musika- 
lische Begleitstucke enthalten, uberdenen Mu- 
" sik erzeugt werden kann und die in Echtzeit ge- 
spielt werden konnen und mehrere Rhythmus- 
blocke besitzen, wovon jederfurwenigstens ei- 
ne Note wenigstens einen Zeitpunkt definiert, 
zu dem die Note gespielt werden sollte, und 
zum Speichern wenigstens eines Abschnitts 
der in Echtzeit uber einem vorgegebenen Zeit- 
intervall in der Vergangenheit erzeugten Musik; 
einem Rhythmusgenerator (100), der in Echt- 
zeit: die rhythmusbezogenen Eingangssignale 
vom Eingabemechanismus empfangt; einen 
der Rhythmusblocke (204) aus den Computer- 
speichermedien anhand der rhythmusbezoge- 
nen Eingangssignale wahlt; und einen Befehl 



zum Spielen der Note zu dem durch den ge- 
wahlten Rhythmusblock definierten Zeitpunkt 
ausgibt; 

einer Tonhohen-Wahleinrichtung (108), die in 
s Echtzeit: die tonhohenbezogenen Eingangssi- 

gnale vom Eingabemechanismus empfangt; ei- 
ne geeignete Tonhohe anhand der tonhohen- 
bezogenen Eingangssignale, eines der musi- 
kalischen Begleitstucke und der gespeicherten 
10 Musik wahlt; und die geeignete Tonhohe aus- 

gibt; und 

einem Schallgenerator (102), der in Echtzeit: 
Befehle vom Rhythmusgenerator, Tonhohen 
von der Tonhohen-Wahleinrichtung und das 
15 gewahlte musikalische Begleitstuck empfangt; 

und ein Audiosignal erzeugt, das wenigstens 
die erzeugte Musik reprasentiert. 

2. System nach Anspruch 1 , wobei der Eingabeme- 
20 chanismus (30) einen Joystick (12) mit einer Basis 
und einem beweglichen Griff, den ein Anwender be- 
tatigt, urn die rhythmusbezogenen und tonhohen- 
bezogenen Eingangssignale zu erzeugen, enthalt. 

25 3. System nach Anspruch 2, wobei der Joystick (12) 
die rhythmusbezogenen Eingangssignale als Ant- 
wort auf die Anwenderbetatigungen des Griffs langs 
einer ersten vorgegebenen Achse (42, 44) erzeugt 
und der Joystick die tonhohenbezogenen Ein- 

30 gangssignale als Antwort auf die Anwenderbetati- 
gungen des Griffs langs einer zweiten vorgegebe- 
nen Achse (46, 48) erzeugt. 

4. System nach Anspruch 1 , 

35 

wobei der Eingabemechanismus (30) auGer- 
dem auf musikalische Effekte bezogene Ein- 
gangssignale schafft; 

ferner mit einem Generatorfur musikalische Ef- 
40 fekte, der die auf musikalische Effekte bezoge- 

nen Eingangssignale vom Eingabemechanis- 
mus empfangt und die Befehle vom Rhythmus- 
generator (100) oder die Tonhohen von der 
Tonhohen-Wahleinrichtung (108) anhand der 
45 auf musikalische Effekte bezogenen Eingangs- 

signale verandert; und 

wobei der Schallgenerator (120) die verander- 
ten Befehle und Tonhohen empfangt und das 
Audiosignal erzeugt, das wenigstens die er- 
50 zeugte Musik mit musikalischen Effekten repra- 

sentiert. 

5. System nach Anspruch 4, wobei der Eingabeme- 
chanismus (30) einen Joystick (12) mit einer Basis, 

55 einem beweglichen Griff und wenigstens einem 
Knopf (50, 52, 54) enthalt. 

6. System nach Anspruch 5, wobei der Joystick (12) 
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die rhythmusbezogenen Eingangssignale als Ant- 
wort auf Anwenderbetatigungen des Griffs langs ei- 
ner ersten vorgegebenen Achse (42, 44) erzeugt, 
wobei der Joystick die tonhohenbezogenen Ein- 
gangssignale als Antwort auf die Anwenderbetati- 
gungen des Griffs langs einer zweiten vorgegebe- 
nen Achse (46, 48) erzeugt und wobei der Joystick 
die auf musikalische Effekte bezogenen Eingangs- 
signale als Antwort auf die Anwenderbetatigungen 
des Knopfes (50, 52, 54) erzeugt. 

7. System nach Anspruch 1 , wobei der Rhythmusge- 
nerator (100) den Befehl zum Spielen der Note zu 
dem durch den gewahlten Rhythmusblockdefinier- 
ten Zeitpunkt, wahrend einer voreingestellten Dau- 
er und mit einer voreingestellten Lautstarke aus- 
gibt. 

8. System nach Anspruch 1, wobei: 

wenigstens einer der mehreren Rhythmusblok- 
ke in dem einen Oder den mehreren Commpu- 
terspeichermedien (22, 36, 38) ferner fur we- 
nigstens eine Note definiert, wie lange die Note 
dauern sollte, wenn sie gespielt wird, und wie 
laut die Note gespielt werden sollte; und 
der Befehl, der vom Rhythmusgenerator (100) 
ausgegeben wird, den Zeitpunkt, zu dem die 
Note gespielt werden sollte, und die Notenlan- 
ge sowie die Lautstarke wie durch den Rhyth- 
musblock definiert angibt. 

9. System nach Anspruch 1 , ferner mit wenigstens ei- 
nem Lautsprecher (18) zum Aussenden des Audio- 
signals. 

10. System nach Anspruch 9, ferner mit einem Verstar- 
ker (18) zum Verstarken des Audiosignals, bevor es 
fur den Lautsprecher zum Aussenden bereitgestellt 
wird. 

11 . System nach Anspruch 1 0, wobei ein programmier- 
ter Computer (20) die Funktionen des Rhythmus- 
generators (100) und der Tonhohen-Wahleinrich- 
tung (108) ausfuhrt und der Verstarker und der 
Lautsprecher (18) sich in dem Computer befinden. 

12. System nach Anspruch 11, wobei der programmier- 
te Computer (20) auGerdem die Funktionen des 
Schallgenerators (102) ausfuhrt. 

13. System nach Anspruch 1 , wobei der Schallgenera- 
tor (102) einen MIDI-Tongenerator enthalt. 

14. System nach Anspruch 1, wobei ein programmier- 
ter Computer (20) die Funktionen des Rhythmus- 
generators (100) und der Tonhohen-Wahleinrich- 
tung (108) ausfuhrt. 



1 5. System nach Anspruch 1 4, wobei der programmier- 
te Computer (20) auBerdem die Funktionen des 
Schallgenerators ausfuhrt. 

s 16. System nach Anspruch 1 , wobei das eine oder die 
mehreren Computerspeichermedien (20, 36, 38) ei- 
nen Computerspeicher (22) umfassen. 

17. System nach Anspruch 1 , wobei das eine oder die 
10 mehreren Computerspeichermedien (22, 36, 38) ei- 
nen Computerspeicher (22) und ein Computer- 
Festplattenlaufwerk (36) umfassen, wobei das 
Festplattenlaufwerk mehrere wahlbare musikali- 
sche Begleitstucke enthalt, uber denen Musik er- 

75 zeugt und gespielt werden kann. 

18. System nach Anspruch 1 , wobei das eine oder die 
mehreren Computerspeichermedien (22, 36, 38) ei- 
nen Computerspeicher (22) und einen CD-ROM mit 

20 mehreren wahlbaren musikalischen Begleitstuk- 
ken, uber denen Musik erzeugt und gespielt werden 
kann, umfassen. 

19. System zum Erzeugen von Musik nach Anspruch 
25 1, wobei der Eingabemechanismus (30) eine Vor- 

richtung enthalt, die wenigstens eine erste Betati- 
gungsachse (42, 44) und eine zweite Betatigungs- 
achse (46, 48) besitzt, wobei die Eingabevorrich- 
tung rhythmusbezogene Signale als Antwort auf ei- 
30 ne Anwenderbetatigung der Eingabevorrichtung 
langs der ersten Achse schafft und tonhohenbezo- 
gene Signale als Antwort auf die Anwenderbetati- 
gung der Eingabevorrichtung langs der zweiten 
Achse schafft. 

35 

20. System nach Anspruch 19, wobei die Eingabevor- 
richtung (30) einen Joystick (12) mit einem beweg- 
iichen Griff enthalt, den der Anwender langs der er- 
sten Achse (42, 44) betatigt, urn den Joystick zu 

40 veranlassen, die rhythmusbezogenen Signale zu 
schaffen, und langs der zweiten Achse (46, 48) be- 
tatigt, um den Joystick zu veranlassen, die tonho- 
henbezogenen Signale zu schaffen. 

45 21. System nach Anspruch 19, wobei in der Eingabe- 
vorrichtung (30) die ersten und zweiten Achsen 
senkrecht zueinander angeordnet sind. 

22. System nach Anspruch 19, wobei die Eingabevor- 
50 richtung (30) einen kontinuierlichen Controller ent- 
halt. 

23. System nach Anspruch 19, wobei die Eingabevor- 
richtung (30) eine Maus enthalt. 

55 

24. System nach Anspruch 19, wobei die Eingabevor- 
richtung (30) einen Trackball enthalt. 
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25. System nach Anspruch 19, wobei die Eingabevor- 
richtung (30) einen Fader enthalt. 

26. System nach Anspruch 19, wobei die Eingabevor- 
richtung (30) einen Schieber enthalt. 5 

27. System nach Anspruch 19, wobei die Eingabevor- 
richtung (30) einen raumlichen Sensor enthalt. 

28. System nach Anspruch 19, wobei die Eingabevor- 10 
richtung (30) einen diskreten Controller enthalt. 

29. System nach Anspruch 19, wobei die Eingabevor- 
richtung (30) eine Spielkonsole enthalt. 

1$ 

30. System nach Anspruch 19, wobei die Eingabevor- 
richtung (30) mehrere Schalter enthalt. 

31. System nach Anspruch 19, wobei die Eingabevor- 
richtung (30) mehrere Knopf e enthalt. 20 

32. System zum Erzeugen von Musik in Echtzeit nach 
Anspruch 1 , wobei: 

die empfangenen rhythmusbezogenen Signale 25 
als Antwort auf eine Anwenderbetatigung einer 
Eingabevorrichtung (30) langs einer ersten Be- 
tatigungsachse (42, 44) der Eingabevorrich- 
tung geschaffen werden; 

die empfangenen tonhohenbezogenen Signale 30 
als Antwort auf eine Anwenderbetatigung der 
Eingabevorrichtung langs einer zweiten Betati- 
gungsachse (46, 48) der Eingabevorrichtung 
geschaffen werden. 

35 

33. System nach Anspruch 32, wobei: 

die empfangenen rhythmusbezogenen Signale 
als Antwort auf die Anwenderbetatigung der 
Eingabevorrichtung (30) langs der ersten Ach- 40 
se (42, 44), die zur zweiten Achse (46, 48) 
senkrecht ist, geschaffen werden; und 
die empfangenen tonhohenbezogenen Signale 
als Antwort auf die Anwenderbetatigung der 
Eingabevorrichtung langs der zweiten Achse, 45 
die zur ersten Achse senkrecht ist, geschaffen 
werden. 

34. System nach Anspruch 33, wobei: 

so 

die empfangenen rhythmusbezogenen Signale 
als Antwort auf die Anwenderbetatigung eines 
Griffs eines Joysticks (12) langs der ersten 
Achse (42, 44) geschaffen w rden; und 
die empfangenen tonhohenbezogenen Signale 55 
als Antwort auf die Anwenderbetatigung des 
Griffs des Joysticks langs der zweiten Achse 
(46, 48) geschaffen werden. 



R vendications 

1. Systeme pour creer de la musique en temps reel, 
comprenant : 

un mecanisme (30) d'entree qui fournit des si- 
gnaux d'entree relatifs a un rythme et des si- 
gnaux d'entree relatifs a une hauteur ; 
un ou plusieurs supports informatiques (22, 36, 
38) de stockage ayant une pluralite de pistes 
d'accompagnement musical pouvant etre se- 
lectionnees sur lesquelles de la musique peut 
etre creee et jouee en temps reel, ayant une 
pluralite de blocs de rythme dans lesquels cha- 
que bloc definit, pour au moins une note, au 
moins un instant auquel la note devrait etre 
jouee, et pour stocker au moins une portion de 
la musique creee en temps reel pendant un in- 
tervals predetermine de temps dans le passe ; 
un generateur (100) de rythmes qui, en temps 
reel : regoit les signaux d'entree relatifs a un 
rythme du mecanisme d'entree ; selectionne 
Tun des blocs (204) de rythme dans le support 
informatique de stockage en se basant sur les 
signaux d'entree relatifs a un rythme ; et emet 
une instruction de jouer la note a I'instant defini 
par le bloc selectionne de rythme ; 
un selectionneur (108) de hauteur qui, en 
temps reel : regoit les signaux d'entree relatifs 
a une hauteur du mecanisme d'entree ; selec- 
tionne une hauteur appropriee en se basant sur 
les signaux d'entree relatifs k une hauteur, I'une 
selectionnee des pistes d'accompagnement 
musical, et la musique stockee ; et emet la hau- 
teur appropriee ; et 

un generateur (102) de sons qui, en temps 
reel : re?oit des instructions du generateur de 
rythmes, des hauteurs du selectionneur de 
hauteur, et la piste selectionnee d'accompa- 
gnement musical ; et produit un signal audiofre- 
quence representant au moins la musique 
creee. 

2. Systeme selon la revendication 1, dans lequel le 
mecanisme (30) d'entree comprend un manche 
(12) a balai ayant une base et une poignee mobile 
qu'un utilisateur manipule pour creer les signaux 
d'entree relatifs k un rythme et relatifs k une hau- 
teur. 

3. Systeme selon la revendication 2, dans lequel le 
manche (12) a balai cree les signaux d'entree rela- 
tifs a un rythme en reponse a des manipulations par 
I'utilisateur de la poignee le long d'un premier axe 
predetermine (42, 44), et le manche k balai produit 
les signaux d'entree relatifs k une hauteur en repon- 
se a des manipulations par I'utilisateur de la poi- 
gnee le long d'un deuxieme axe predetermine (46,* 
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48). 

4. Systeme selon la revendication 1 : 

dans lequel le mecanisme (30) d'entree fournit 
aussi des signaux d'entree relatifs a des effets 
musicaux ; 

comprenant en outre un generateur d'effets 
musicaux qui recoit les signaux d'entree relatifs 
a des effets musicaux du mecanisme d'entree 
et qui modifie les instructions du generateur 
(100) de rythmes ou les hauteurs du selecteur 
(108) de hauteur en se basant sur les signaux 
d'entree relatifs a des effets musicaux ; et 
dans lequel le generateur (102) de sons regoit 
les. instructions et hauteurs modifiees et cree le 
signal audiofrequence qui represente au moins 
la musique creee avec des effets musicaux. 

5. Systeme selon la revendication 4, dans lequel le 
mecanisme (30) d'entree comprend un manche 
(12) a balai ayant une base, une poignee mobile et 
au moins un bouton (50, 52, 54). 

6. Systeme selon la revendication 5, dans lequel le 
manche ( 1 2) & balai produit les signaux d'entree re- 
latifs a un rythme en reponse a des manipulations 
par I'utiiisateur de la poignee le long d'un premier 
axe predetermine (42, 44), le manche a balai pro- 
duit les signaux d'entree relatifs a une hauteur en 
reponse & des manipulations par I'utiiisateur de la 
poignee le long d'un deuxieme axe predetermine 
(46, 48), et le manche a balai produit les signaux 
d'entree relatifs a des effets musicaux en reponse 
a des manipulations par I'utiiisateur du bouton (50, 
52, 54). 

7. Systeme selon la revendication 1 , dans lequel le ge- 
nerateur (100) de rythmes emet I'instruction de 
jouer la note k I'instant defini par le bloc selectionne 
de rythme, et pour une duree par defaut et avec une 
intensite par defaut. 

8. Systeme selon la revendication 1, dans lequel : 

au moins un de la pluralite de blocs de rythme 
dans le ou les supports informatiques (22, 36, 
38) de stockage definit en outre, pour au moins 
une note, le temps que la note devrait durer 
quand elle est jouee et avec quelle intensite la 
note devrait etre jouee ; et 
I'instruction emise par le generateur (100) de 
rythmes indique I'instant auquel la note devrait 
etre jouee et la duree et I'intensite de la note 
telles que definies par le bloc de rythme. 

9. Systeme selon la revendication 1 , comprenant en 
outre au moins un haut-parleur (18) pourdiffuser le 



signal audiofrequence. 

10. Systeme selon la revendication 9, comprenant en 
outre un amplificateur (18) pour amplifier le signal 

s audiofrequence avant qu'il soit fourni au haut- 
parleur pour etre diffuse. 

11. Systeme selon la revendication 10, dans lequel un 
calculateur programme (20) execute les fonctions 

io du generateur (100) de rythmes et du selecteur 
(1 08) de hauteur, et I'amplificateur et le haut-parleur 
(18) sont incorpores au calculateur. 

12. Systeme selon la revendication 10, dans lequel le 
is calculateur programme (20) execute aussi les fonc- 
tions du generateur (102) de sons. 

1 3. Systeme selon la revendication 1 , dans lequel le ge- 
nerateur (1 02) de sons comprend un g6n6rateur de 

20 sons MIDI. 

14. Systeme selon la revendication 1, dans lequel un 
calculateur programme (20) execute les fonctions 
du generateur (100) de rythmes et du selecteur 

25 (108) de hauteur. 

15. Systeme selon la revendication 14, dans lequel le 
calculateur programme (20) execute aussi les fonc- 
tions du generateur (102) de sons. 

30 

16. Systeme selon la revendication 1 , dans lequel le ou 
les supports informatiques (22, 36, 38) de stockage 
comprend une memoire informatique (22). 

35 17. Systeme selon la revendication 1 , dans lequel le 
ou les supports informatiques (22, 36, 38) de stoc- 
kage comprennent une memoire informatique (22) 
et un disque dur informatique (36), le disque dur 
ayant la pluralite de pistes d'accompagnement mu- 

40 sical pouvant etre selectionnees sur lesquelles de 
la musique peut etre creee et jouee. 

18. Systeme selon la revendication 1 , dans lequel le 
ou les supports informatiques (22, 36, 38) de stoc- 
ks kage comprennent une memoire informatique (22) 

et un lecteurde CD-ROM ayant la pluralite de pistes 
d'accompagnement musical pouvant etre selec- 
tionnees sur lesquelles de la musique peut etre 
creee et jouee. 

so 

19. Systeme pour creer de la musique selon la reven- 
dication 1 , dans lequel le mecanisme (30) d'entree 
comprend un dispositif ayant au moins un premier 
axe (42, 44) de manipulation et un deuxieme axe 

55 (46, 48) de manipulation, le dispositif d'entree four- 
nissant des signaux relatifs k un rythme en reponse 
k une manipulation par un utilisateur du dispositif 
d'entree par rapport au premier axe et fournissant 
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des signaux relatifs a une hauteur en reponse a une 
manipulation par I'utilisateur du dispositif d'entree 
par rapport au deuxieme axe. 

20. Systeme selon la revendication 19, dans lequel le 5 
dispositif (30) d'entree comprend un manche (12) a 
balai ayant une poignee mobile qu'un utilisateur 
manipule le long du premier axe (42, 44) pour ame- 
ner le manche k balai a produire les signaux d'en- 
tree relatifs a un rythme, et le long du deuxieme axe to 
(46, 48) pour amener le manche a balai a produire 

les signaux d'entree relatifs k une hauteur. 

21. Systeme selon la revendication 19, dans lequel le 
-—-dispositif (30) d'entree a les premier et deuxieme 

axes disposes perpendiculairement I'un a I'autre. 

22. Systeme selon la revendication 19, dans lequel le 
dispositif (30) d'entree comprend un dispositif con- 
tinu de commande. 20 

23. Systeme selon la revendication 19, dans lequel le 
dispositif (30) d'entree comprend une souris. 

24. Systeme selon la revendication 19, dans lequel le 25 
dispositif (30) d'entree comprend une boule de 
pointage. 

25. Systeme selon la revendication 19, dans lequel le 
dispositif (30) d'entree comprend un potentiometre. 30 

26. Systeme selon la revendication 19, dans lequel le 
dispositif (30) d'entree comprend un curseur. 

27. Systeme selon la revendication 19, dans lequel le 35 
dispositif (30) d'entree comprend un capteur spa- 
tial. 



nis en reponse a une manipulation par I'utilisa- 
teur d'un dispositif (30) d'entree par rapport a 
un premier axe (42, 44) de manipulation du dis- 
positif d'entree ; 

les signaux regus relatifs a une hauteur sont 
fournis en reponse a une manipulation par I'uti- 
lisateur du dispositif d'entree par rapport k un 
deuxieme axe (46, 48) de manipulation du dis- 
positif d'entree. 

33. Systeme selon la revendication 32, dans lequel : 

les signaux regus relatifs a un rythme sont four- 
nis en reponse a une manipulation par I'utilisa- 
teur du dispositif (30) d'entree le long du pre- 
mier axe (42, 44) qui est perpendiculaire au 
deuxieme axe (46, 48) ; et 
les signaux regus relatifs a une hauteur sont 
fournis en reponse k une manipulation par I'uti- 
lisateur du dispositif d'entree le long du deuxie- 
me axe qui est perpendiculaire au premier axe. 

34. Systeme selon la revendication 33, dans lequel : 

les signaux regus relatifs a un rythme sont four- 
nis en reponse a une manipulation par I'utilisa- 
teur d'une poignee d'un manche (12) a balai le 
long d'un premier axe (42, 44) ; et 
les signaux regus relatifs k une hauteur sont 
fournis en reponse k une manipulation par I'uti- 
lisateur de la poignee du manche a balai le long 
d'un deuxieme axe (46, 48). 



28. Systeme selon la revendication 19, dans lequel le 
dispositif (30) d'entree comprend un dispositif dis- 40 
cret de commande. 



29. Systeme selon la revendication 19, dans lequel le 
dispositif (30) d'entree comprend une manette de 
jeu. 45 

30. Systeme selon la revendication 19, dans lequel le 
dispositif (30) d'entree comprend une pluralite d'in- 
terrupteurs. 

so 

31. Systeme selon la revendication 19, dans lequel le 
dispositif (30) d'entree comprend une pluralite de 
boutons. 

32. Systeme pour creer de la musique selon la reven- 55 
dication 1 , dans lequel : 

les signaux regus relatifs k un rythme sont four- 



